Large area databases are used for out-the-window scenes in aircraft simulators. Typically such out-the-window visual systems incorporate high-resolution image insets combined with a generic or low resolution background. In other words, high-resolution images are provided for critical landcover scenes or objects such as buildings or cities and the remaining scenes are filled in with low-resolution graphics or crude polygon images. It will be appreciated that the out-the-window scenes are used to provide the trainee with as life-like experience as possible so as to enhance the training simulation. Low-resolution background images can in fact detract from the training experience and result in negative training for the trainee. Such simulators are commonly used for training aircraft pilots wherein the out-the-window scenes are combined with instrumentation used for that particular aircraft that the pilot is training for. The primary obstacle to greater use of high-resolution imagery in such simulators is the prohibitive acquisition costs of the database imagery.
Such large area databases for out-the-window visual systems typically cover 500 square kilometers. Accordingly, obtaining a high-resolution color image of such a large area would require one to hire a plane or obtain a satellite image in which the resolution is approximately ½ meter to 1.5 meters for each pixel of the image. Such a high resolution image is obtained from 35 millimeter photographs which are quite expensive to obtain for such a large area. Alternatives to photograph images can be obtained from IKONOS™ or Quick Bird™ satellite images which are digital images that are fused with natural color to provide a color image. But, these are also quite expensive. Other satellite images of a large area may also be obtained, but these typically have a resolution in the range of 25–30 meters per pixel. Accordingly, these images do not provide the high-definition required by trainees to provide an optimum training experience. Moreover, such out-the-window visual scenes must incorporate scenes that facilitate simulated take off and landing, low-altitude attacks, high-altitude attacks, bomb damage assessment and in-route operations.
Cost constraints often preclude acquiring high-resolution natural color images from aerial surveys or from satellites with high-resolution multi-band sensors. The usual solution to this high cost is the use of “pan-sharpening,” which is a procedure which fuses high-resolution panchromatic imagery with low-resolution color imagery to produce high-resolution color imagery. Those skilled in the art will appreciate that panchromatic images are those that are sensitive to light of all colors in the visible spectrum. In other words, these are black and white images with the appropriate gray scale levels. Panchromatic images are commercially available at a low cost from the United States Geological Survey.
The most widely available and lowest cost source for low-resolution color imagery is Landsat's™ multi-spectral bands. These bands do not correspond exactly to the wavelengths of red, green and blue visible light and, as such, unprocessed Landsat™ imagery does not look “natural.” Also the information content of the Landsat near-blue band is degraded because of atmosphere absorption at that wavelength. For both of these reasons, it is necessary to transform the Landsat multi-spectral bands to natural colors.
The standard technique for synthesis of natural color imagery is the application of a mathematical transform—usually simple image stretching and biasing—to a selected set of three of the Landsat™ bands, thus producing the three natural color red, green and blue bands. The difficulty is that no transform has been found that generates acceptable results over large areas because of the variability of the source imagery. The result is that natural color is provided for some areas and non-natural or false colors are provided for other areas. Additionally, for cases where aerial photography has been acquired for small areas, it is difficult to color-match a synthetic pan-sharpened imagery and non-synthetic imagery. In other words, transform or fusing processes result in an image that includes colors which are not naturally occurring. For example, such colors include the bright neon oranges or greens that are sometimes seen on traffic signs, but which are never encountered in out-the-window visual scenes viewed by aircraft pilots.
It will be appreciated that the existing prior art techniques to generate synthesized color imagery require the use of specified constants or “magic numbers” that are used by the stretch/bias transforms. Because of the variability of the multi-spectral imagery, it is difficult to specify constants that work acceptably over large areas. The usual result from such a process is an output consisting of both natural colors—trees and vegetation—and non-natural colors, for example, the previously referred to neon colors.
Difficulties also arise in combining the multi-spectral imagery that the “sharpening” steps used. In the past the sharpening or fusing steps required that histograms be taken of the entire image and also of third level segments without consideration of surrounding areas. In other words, a histogram would need to be taken for the entire area, another histogram taken of a smaller area and yet another series of histograms for smaller areas within the entire image area. These different levels of histograms further introduce color discontinuities that cannot be correlated when all of the images are combined. Moreover, taking so many different histograms and re-combining them uses valuable computing time in rendering the image.